Add Filebeat module for monitoring Santa #9540

andrewkroh · 2018-12-14T07:18:11Z

This adds a Filebeat module for monitoring and parsing the log file generated
by the Google Santa, a process monitoring tool for MacOS. This module includes
an overview dashboard.

https://github.com/google/santa

ruflin

🎅 Overall LGTM. I wonder if that is a module we should keep in the long term? ;-)

ruflin · 2018-12-14T07:30:03Z

filebeat/module/santa/_meta/fields.yml

+  description: >
+    Santa Module
+  fields:
+    - name: santa


I assume all fields are on the module level on purpose?

Yeah, I'm never expecting there to be anything other than the "log" dataset so dropped the .log. Hopefully that's ok.

webmat

LGTM.

Suggesting a minor improvement to make conversion to int easier, and also convert user/group.id to int.

webmat · 2018-12-14T14:01:24Z

filebeat/module/santa/log/ingest/pipeline.json

+            "grok": {
+                "field": "message",
+                "patterns": [
+                    "\\[%{TIMESTAMP_ISO8601:process.start}\\] I santad: action=%{NOT_SEPARATOR:santa.action}\\|decision=%{NOT_SEPARATOR:santa.decision}\\|reason=%{NOT_SEPARATOR:santa.reason}\\|sha256=%{NOT_SEPARATOR:hash.sha256}\\|path=%{NOT_SEPARATOR:process.executable}(\\|args=%{NOT_SEPARATOR:process.args})?(\\|cert_sha256=%{NOT_SEPARATOR:certificate.sha256})?(\\|cert_cn=%{NOT_SEPARATOR:certificate.common_name})?\\|pid=%{NUMBER:process.pid}\\|ppid=%{NUMBER:process.ppid}\\|uid=%{NUMBER:user.id}\\|user=%{NOT_SEPARATOR:user.name}\\|gid=%{NUMBER:group.id}\\|group=%{NOT_SEPARATOR:group.name}\\|mode=%{WORD:santa.mode}"


Suggested change

"\\[%{TIMESTAMP_ISO8601:process.start}\\] I santad: action=%{NOT_SEPARATOR:santa.action}\\|decision=%{NOT_SEPARATOR:santa.decision}\\|reason=%{NOT_SEPARATOR:santa.reason}\\|sha256=%{NOT_SEPARATOR:hash.sha256}\\|path=%{NOT_SEPARATOR:process.executable}(\\|args=%{NOT_SEPARATOR:process.args})?(\\|cert_sha256=%{NOT_SEPARATOR:certificate.sha256})?(\\|cert_cn=%{NOT_SEPARATOR:certificate.common_name})?\\|pid=%{NUMBER:process.pid}\\|ppid=%{NUMBER:process.ppid}\\|uid=%{NUMBER:user.id}\\|user=%{NOT_SEPARATOR:user.name}\\|gid=%{NUMBER:group.id}\\|group=%{NOT_SEPARATOR:group.name}\\|mode=%{WORD:santa.mode}"

"\\[%{TIMESTAMP_ISO8601:process.start}\\] I santad: action=%{NOT_SEPARATOR:santa.action}\\|decision=%{NOT_SEPARATOR:santa.decision}\\|reason=%{NOT_SEPARATOR:santa.reason}\\|sha256=%{NOT_SEPARATOR:hash.sha256}\\|path=%{NOT_SEPARATOR:process.executable}(\\|args=%{NOT_SEPARATOR:process.args})?(\\|cert_sha256=%{NOT_SEPARATOR:certificate.sha256})?(\\|cert_cn=%{NOT_SEPARATOR:certificate.common_name})?\\|pid=%{NUMBER:process.pid:int}\\|ppid=%{NUMBER:process.ppid:int}\\|uid=%{NUMBER:user.id:int}\\|user=%{NOT_SEPARATOR:user.name}\\|gid=%{NUMBER:group.id:int}\\|group=%{NOT_SEPARATOR:group.name}\\|mode=%{WORD:santa.mode}"

Suggestion to simplify the pipeline, and remove the need for a few convert blocks below.

Notice that I'm also converting the user.id and group.id in this suggestion.

Updated as suggested, but I did not change user.id or group.id because they are keyword in ECS (I assume this is in order to accommodate SID values from Windows).

webmat · 2018-12-14T14:08:25Z

filebeat/module/santa/_meta/fields.yml

+      description: SHA256 hash of code signing certificate.
+
+    # Auditbeat FIM is using this field for the same purpose.
+    - name: hash.sha256


Is it common elsewhere to use certificate and keyword at the top level?

The closest thing we have is tls.{client,server}_certificate.fingerprint.sha256.

webmat · 2018-12-14T14:10:54Z

@ruflin The project has been alive for about 4 years. I doubt it's a holiday prank LOL.

This adds a Filebeat module for monitoring and parsing the log file generated by the Google Santa, a process monitoring tools for MacOS. This module includes an overview dashboard. https://github.com/google/santa

jsoriano · 2018-12-17T12:28:15Z

Oh when I read the title I thought this was going to be an easter egg! 😄 🎅

elasticmachine · 2018-12-17T12:52:13Z

Pinging @elastic/secops

ruflin

LGTM. Let's backport to 6.x?

andrewkroh · 2018-12-18T03:42:36Z

It can be backported. I'll add the label as a reminder. I'm pretty sure the backport will need some fields added to fields.yml since this currently depends on some ECS fields.

MikePaquette · 2019-03-21T04:54:34Z

@andrewkroh should this be included in our announcement of features in the 7.0 release? I don't think we've mentioned it anywhere yet. Should it be listed as ECS-compatible? Thanks.

andrewkroh · 2019-03-21T14:39:49Z

@MikePaquette Absolutely. And it is ECS-compatible.

andrewkroh added module review Filebeat Filebeat labels Dec 14, 2018

andrewkroh force-pushed the feature/fb/santa-module branch from f2f1caf to e3cd5e9 Compare December 14, 2018 07:19

ruflin reviewed Dec 14, 2018

View reviewed changes

webmat approved these changes Dec 14, 2018

View reviewed changes

andrewkroh requested a review from a team as a code owner December 17, 2018 03:13

andrewkroh added 2 commits December 16, 2018 22:21

Add Filebeat module for monitoring Santa

5dd44e8

This adds a Filebeat module for monitoring and parsing the log file generated by the Google Santa, a process monitoring tools for MacOS. This module includes an overview dashboard. https://github.com/google/santa

Make pid and ppid integers in grok pattern

0a6362a

andrewkroh force-pushed the feature/fb/santa-module branch from f773e33 to 0a6362a Compare December 17, 2018 03:22

andrewkroh added 2 commits December 16, 2018 23:28

Add grok pattern for action=DISKAPPEAR

d7496ca

make update for santa.disk fields

66f4f79

jsoriano approved these changes Dec 17, 2018

View reviewed changes

tsg added the SecOps label Dec 17, 2018

ruflin approved these changes Dec 17, 2018

View reviewed changes

andrewkroh merged commit 4962a59 into elastic:master Dec 17, 2018

andrewkroh added the needs_backport PR is waiting to be backported to other branches. label Dec 18, 2018

cwurm mentioned this pull request Feb 6, 2019

[Filebeat] Remove Santa module directory accidentally backported to 6.x #10610

Merged

andrewkroh removed the needs_backport PR is waiting to be backported to other branches. label Aug 29, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add Filebeat module for monitoring Santa #9540

Add Filebeat module for monitoring Santa #9540

andrewkroh commented Dec 14, 2018 •

edited

Loading

ruflin left a comment

ruflin Dec 14, 2018

andrewkroh Dec 14, 2018

webmat left a comment

webmat Dec 14, 2018

webmat Dec 14, 2018

andrewkroh Dec 17, 2018

webmat Dec 14, 2018

andrewkroh Dec 17, 2018

webmat commented Dec 14, 2018

jsoriano commented Dec 17, 2018 •

edited

Loading

elasticmachine commented Dec 17, 2018

ruflin left a comment

andrewkroh commented Dec 18, 2018

MikePaquette commented Mar 21, 2019

andrewkroh commented Mar 21, 2019

Add Filebeat module for monitoring Santa #9540

Add Filebeat module for monitoring Santa #9540

Conversation

andrewkroh commented Dec 14, 2018 • edited Loading

ruflin left a comment

Choose a reason for hiding this comment

ruflin Dec 14, 2018

Choose a reason for hiding this comment

andrewkroh Dec 14, 2018

Choose a reason for hiding this comment

webmat left a comment

Choose a reason for hiding this comment

webmat Dec 14, 2018

Choose a reason for hiding this comment

webmat Dec 14, 2018

Choose a reason for hiding this comment

andrewkroh Dec 17, 2018

Choose a reason for hiding this comment

webmat Dec 14, 2018

Choose a reason for hiding this comment

andrewkroh Dec 17, 2018

Choose a reason for hiding this comment

webmat commented Dec 14, 2018

jsoriano commented Dec 17, 2018 • edited Loading

elasticmachine commented Dec 17, 2018

ruflin left a comment

Choose a reason for hiding this comment

andrewkroh commented Dec 18, 2018

MikePaquette commented Mar 21, 2019

andrewkroh commented Mar 21, 2019

andrewkroh commented Dec 14, 2018 •

edited

Loading

jsoriano commented Dec 17, 2018 •

edited

Loading